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ABSTB ACT 

The course in parsing English is essentially a survey 
and comparison -of several of the principal systems used for 
understanding natural language. The basic procedure of parsing is 
described. The discussion of the principal systems is based on the 
idea that "meaning is procedures," thrat is, that the procedures of 
application give a parsed structure its significance. Natural 
language systems should be content- rather than structure-motivated, 
i.e. they should be concerned with linguistic problems revealed by 
parsing rather than with the relation of the proposed structure of 
the system t« the structures of other system^. Sithin this framework, 
«inograd»s understanding system, SHBDLO, is described and discussed, 
as are the second generation systems of Simmons, Schank, Colby and 
Wilks. A subseguent discussion compares all these syBteis. Conclu^ng 
remarks outline immediate problems, including the need for a good^ 
memory model and the use of texts, rather than individual example 
sentences,, for investigation. (CLK) 
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SYNOPSIS/NOTES rdR COURSE PARSING ENGLISH 
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1. Introduction 

This course will be, in essence, a survey and compari- 
son cf several of the principal systems for understanding 
natural language now under discussion in the Artificial 
Intelligence (AI) community . The dimensions of compart*^ 
son chosen will reflect the presuppositions and prejudices 
of the lecturer, and I shall try to make some of these 
clear right at the beginning. 

Given any proposed structure for natural language what- 
ever, we may distinguish between the structure itself, » 
and its correlation* with pieces of language* Among such 
Correlations we may distinguish between parsings and 
assignments . By "parsing" here I mean the provision of 
definite procedures of application , and by assignment 
I mean the provision of no more thsn a list of correspon- 
dences, between cYfUnVin of language and formal structures. 

Logicians often tell us that, say, the structure of the 

I 

sentence **John loves his wife's sister" is of the form 
Ex.Ey .Ez : x«J, y«^M J), z«S (W (J) ) . L (X 2),but they rarely, 
if ever, provide a parsing of that structure onto the 
sehtence, for the procedure is considered obvious,. That 
is what I mean by an assignment, or the provision'of a 
list of correspondences between sentences and stt^uc^^ures • 

At this point let me try and say a little what 
parsing is rather than what it is not. Let us coji^ider 
the structure specified by one of the simplest grammars : 
a content-free phrase structure grammar. This might con- 
tain rules like the following (an example of Winograd's): 

S * — ^NP VP 

NP — ^ Determiner Noun 
VP^ — SVerb NP 

Determiner ^ a, the 

Noun ^ rabbit, cucumber 

Verb ^ nibbled 
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Sentence : The rabbit nibbled a cucumber . 
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This sentence "The rabbit nibbled the cucumber" is 
considered to be produced by the listed rewrj^te rules of 
this grammar. The grammar is though^ of a implicitly 
defining a procedure*, starting with /the synA)olS, and re- 
writing it as the right hand side o^ the x/ixle containing 
it, and then rewriting those symbols by o£her rules (or 
the same r*ule can be used again if it is^ possible to do 
so) , as long as only one symbol jik rewritten each time 
a rule is applied. The production of this sentence is 
usuallyT^B^resented by a phrase-itructure tree as follows 
(also called a phrase-marker) whbre ev^ry production of 
branches from a node correspondaj to the application of 

I i 

one^ of these rewri te rules « A is j just described , eventually 
symbols are written that arc a<j!jbually words of English, 
and the process can go no furthler when every node is 
an English word. 

S 




Determiner Noun Verb 



Noun 
Cucumber 



grtie Rabbit Nibbled 

') 

, ' • (' 
/ I- 

Bowevl^r, this process of idealised sentence generation 
is n^t a parsing process f^r that is always from the 
sen^nce and some such battery of rules to a structure 
liki that of the d|.agram. The structure obtained is thei 
caljiLed a parsing because it reflects certain syntactic 
rel/ationships that we may think of as holding between tfxe 



wo|rds of the sentence. Thus in the tree structure, 

ha^ been tied to "rabbit" by the application of the ri^le 

N^lf — ^ Determiner Noun and that relation of "the"^ to "Arabbit 

& / 
i| dependence of a determiner on the lou* it d%tyermines 
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For the purpose of parsing we thjfnk of each word in the 
sentence to be parsed as attached to one or nore gramma- 
tical categories: in this case as we see from the rules 
••rabbit" is attached to a single category '•Noun" . There 
are two ways of doing the parsing: top-down and bottom^up* 
Bottom- up is the more straightforward way , and is illustra- 
ted by the next figure (all figures in this part are due 
to Winograd) * The words of the sentence are listed and' 

L 

starting from the left pf the sentence, we attempt to 
replace each one by its srategory, and then to rewrite 
successively pairs of category symbols by reversing the 
grammar * s rewrite rules j, until we reach the final sentence 
symbol S* The lines of the derivation (the tree above 
upside downt in fact) can then be considered as the 
parsing. | 

The rabKit nibbled a cucumber. 

Determiner rabbity nibbled a cucumber. 

Determiner. Noun nibbled a cucumber. 

NP nibbled a fcucumber 

1 

NP Verb a cuC|timber 

NP Verb Deter|Diner cucumber . 

NP Verb Determiner Noun 

NP verb NP 1 

^ NP VP \ I 

\ r 

s 

Top down (yarsing 3tart|s, not with the words and their 
categories, bu± with generations as described initially 

form the top^^^in^e of the' grammar starting with symbol S. 

1 

It continues to generate aequences of categories until 
it finds one that will mat^ch directly onto the leftmost 
word of the sentence , "The!" . This procedure continues , 



1 

\ 

\ 



ERIC 



■■■J 



4. 



as illustrated by the next diagram, until the last word at 
the right is reached andv then once again the lines of the 
procedure are the parsini. The two derivations aboye 
(bottom up and top dc^wn/ are formally equivalent. - 



Looking for: S 

Looking for: NP | VP 

Looking for: Determiner^ Noun VP 

Looking for: Noun \ VP 

Looking for: VP 

Looking for: Verb | NP 

Looking for: NP 

Looking for: Determiner | Novin 

Looking for: Noun 

Looking for: 



I the tabbit n^tbled a cucumber. 
I the rabbit nibbled a cucxmber. 
ithe rabbit nibbled a cucumber. 

the I rabbit nibbled a cucuaber. 

the rabbit J nibbled a cucumber ♦ 

th^ rabbit | nibbled a cucumber. 

the rabSit nibbled [ a cucumber. 

the rabbit fiibbled \ a^j cucumber. 

the rabbit nibbled a | cucumber. 

the rabbit nibbled a cucumber j . 



Several things should be noted here: parses do not have 
to be left to right , they could easily be the other way round. 
The structure is rarely as trivial as this one, and usually a 
word will have several possible categories assigned to it in 
the dictionary and part of the job of the parsing process is 
to find out which of those roles the word has in any parti-- 
cular sentence. So ''run" has a Verb category in "I run for a 
bus** but a Ncun category in " I built a new chicken run** , (a 
run is a cage for chickens). So, if we used either of the 
above processes to parse "I run for a bus", we would find that 
we produce the correct structure: 



J 



I 
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NP VP 

/ /\ 



N VP 

I I 

V 

' I 

I run 




but never a structure where " run" had its N (Noun) category 
because there is no sequf?nce of proper grammar rule appli^ 
cations that cobfd produce a tree for that? sentence with 
the node above "run" labelled N. ^tn-^^^rticuXar , there is 
a rule: 

VP — ^V (and then V-JSL-^ run) ^ 

# 

but no rul«: 

VP ^ r: (and then N >run) 

So a "reading" of "runT is excluded bz the parsing of 
this sentence* 

At this point, another important pair of technical 
terms enter: breadth-first and depth-first parsings. 
Breadth first is the parallel treatment of all possible 
alternative structures at a given time, none is given 
any precedence. They are all developed until one or more 
reach the success symbol S (if the parsing is bottom-'up) . 
In depth-first parses the alternative structures are 
treates sequentially, and if one combination is no longer 
possible success (because, in this grammar no more re- 
writes of a string are possible even though the parsing 
has not been completed), then the system must back-track, 
which means going back and continuing, one of the alter- 
native structures that h^d not yet been te-ited. 
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In the case of transformational structures in particular, 
a sentence .is often pirsed so as to produce a structure 
quite different from the, original form of the sentence. 
So, to take a classic example; a passive sentence like* 
••I was given the money last week** might be given a parsed 
structure representing it whose form was actually that of 
"Someone gave me the money last week", as regards the order 
of its principal constituents. So we may say that a 
parsing may produce a structure quite different from the 
superficial form ot the input sentence. Grammatical ^ 
parsings programs of this sor^^ere a growth industry 
in the sixties, but I do not want to go into them in any 
detail here beyond the introduction of the basic notions. 
That is because one of the assumptions of this course is 
that grammatical (or syntactic) parsing of the sort de-- 
scribed is not fundamental^ and that is need not be even 
a preliminary to assigning a useful meaning structure to 
sentences. However; I |Jhall argue that pnly indirectly, 
and much later on. I shall go into some detail describing 
the syntactic parser behind Terry winogra^'s program. 
Later, wh<ni I come to my own /system (and that of Chris 
RiesbeqX) I shall describe parsing procedures (that is 
to s/ty procedural ascription of structures to sentences) 
vfh^e the structures are not syntactic at all, but 
Xsemantic, and at thfit point I shall deal with some of 
the difficulties of manipulating syntactic information 
by quJLte other means. 

Let us return for a moment to what I called assignments . 
An intermediate case betxveen parsings and assignments 
would be when a sentence was prcSf^rrod as its own repre- 
sentation, so that its structure was no more than what 



a reader of the sentence would intuitively assign to it, 
were he asked to do so. That is to sa^j^ if any writer 
were to offer us ''John loves his wife's sister** as the 
representation of ''John loves his wife * s sister** # we would 
know something odd was going on. Things are rarely as 
bad as thiSf but they do approach this situatiorif one might 
say , asymptotically . So for example , in any A I language 
program in which the representation for "John is at the 
station- is given as (AT JOHN STATION) , and this is quite 
a common move as you will see from other courses in this 
tutorial, then we are being given a structure which is 
only trivially different from that of the sentence itself, 
and the reader^ of such a structure must "parse" it 
intuitively by some direct matching such ^as : 

AT JOHN STATION 

I 

\ 

^ 




is at '^the station 

I would claim that such a process is aliifo not a serious 
parsing, but rather what I would call a prd^^oction , in that 
the sentences chosen as examples arc cssent^^ally pro j ec- 
tions of the representation itself, and moreover, and this 

is the important point, the representation is trivial 

I 

and can only bo understood from our knowledge of the 

1 

original sentence. If tihe representation and sentence were 
in a foreign language, Jf which one knew nothing, one 

would soon see that withiout explicit rules of interpret 

i' 

tation for the represent|ition (which we are not normally 
given) one could not understand it. 

A full, complex, and explicit representation can also 
be projected in some sense, of course, but at the moment 
I shall* restrict the term] to simplistic representations. 
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The effect of both assignments and projections is to 
make parsing seem either unnecessary, or" at most a dev 
tachable and ultimately dlspensible **front end" to some 
other system. It will become clear that I think this 
assumption to be radically mistaken , in that , as )l am 
using "parsing", it is not only a front end, and not 
only a test of the value and efficacy of the representa* 
tion parsed, for it also gives meaning to the structurp. 

My view is a version of the "meaning is procedures** 
attitude : that the procedures of its application give 
a parsed structure significance, and that without the 
procedure or the demonstrated possibility of them, the ^ 
structure proposed is essentially meaningless, and the 
plaything of formalists. 

At this point, I shall consider, in passing*, a modero 
movement in ^linguistics , Generative Semantics, which has 
got into a position in which, given its theoretical *pre- / 
suppositions, the structures it postulates cannot , demon* 
strably be parsed onto sentences. I shall argue %his in 
detail and point out that the position is not made any 
less serious by the fact that most of the practitioners 
of Generative Semantics have not noticed it. 

Let me make a second distinction at thi^point that 
will be important in^ the examination of systems that follows 
It is betweei^ Al language systems that are primarily 
content-m'crtivated and those which are ppimarily s4:ruc - 
turally- motivated I shall not claJ^ that systems are of , 
one sort or the other, since there is as always, a con- 
tinuum in these matters, but will ask readers to keep 
the distinction in mind during the description and dis- 
cussion of systems. 
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By structurally-imotivated systems I mean those which, ^ 
whatever their natural language capacities, are set up 
in order to solve problems that are essentially non-lin- 
guistic; and are concerned with the relation of the 
proposed representation of structure to Q>hi5>v structures . 

To put it another structural ly^iroY'V*^®^*y*^®'°* 

not Concerned with the problems posed by--t7ie extraction 
of the content of input natural language, since the nature 
and purpose of the represerttatloj^ adopted is aimed $it the 
problem rather than at language • 

A paradigm of this would be Bobrow's early •'Natural 
Language Sysjjj^ra" (Bobrow '68) that solved .elementary ^ 
algebra problems* It did indeed take in a form of re- 
stricted Bngli'&hr, but it, in my view\, was essentially •c. 
a system to solve algebra problems. \ln contrast to this, 
one cannot "even imagine a machine translation system 
that accepted and prpduced structures i for the^ extraction 
of content/is the problfem tho whole system Is Met up to 

nolve. \ y ^ 

It will be clear that ray distinctions are evaluative 
in nature, and indeed one of the purposes of this course 
will be to argue in an indirect way that natural language 
systems, 1) should be parsed rather than assigned or 
projected and 2) should be content-- rather than structurally 
motivated. I certainly yCannot prove, however, that 
advance with "The problem of natui^al language" will--come 
through content-rather than structurally-motivated systems. 
Notice too, here that the two distinctions are not neces- 
sarily connected: winograd's system, for example, that I 
shall discuss in some detail later, is certainly a parsed 
system, yet is it arguably structurally-motivated. . 
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What do I mean by "the problem of natural language"^? 

i 

I mean the problem that has dogged mechanical language 
analysis * since Its Inception In the Fifties, and which 
centres round the problem of systematic ambiguity . l^et 
me give a brief and^ tendentious historical sketch at this 
point: I take It that the "machine translation era'* of 
the Fifties and early Sixties was a bold atfempt to do 
an enormous taks: extract the content of natural language 
utterances. It failed, and for three reasons: word sense 
ambiguity or polysemy, case ambiguity (the ambiguity of 
prepositiops) and referential ambiguity (the ambiguity of 
pronouns) • It failed to develop and implement structures 
to deal with these features of even non-metaphorical * 
natural language. Generative linguistics was born partly 
in response to this failure: it has provided complex 
structures, but has lost the senso of any definite task 
to be performed, and the most recent sign of this failure 

4 

of nerve has been a retreat from any relation to paradn^, 
towards a totally structurally motivated approach, and 
the doctrine that the true structure of language Is its 
logical structure . 

And so we come to the AI approach, which rcfpresents- a 
return to infinite tasks and procedures and now slowly 
but surely a withdrawal from the structurally-motivated 
approach* Host early work on language within AI was 
highly structurally- motivated: the importance of the 
Predicate Calculus as the basic form of representation 
of langlKge was taken for granted by many leading AI 
workers • What they were interested in was proving theorems 
via the derivation of one predicate calculus representa- 
tion from another, an as far as they were concerned there 
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sloply waj no "problem of language " for the ^ree "intrac- ^ 
tables" of language ambiguity listed above were just ignored. 
In the terms I de.veloped earlier, they really did bj^ieve 
that natural language was a projection of some underJLying 
, logical format, Slov.ly but aUiToly^ reality has crept in. 
Most recoJitly, 5n a survey of AI language systems, Terry 
Winograd (Winograd 'TB) has distinguished between "first* 
and "second" rjeneratiou AT lanQ.aagc systems, modestly 
j^JLacing his own at the *^nd of the first generation. I 
shall not examine his distinction in* any detail, but it 
does, I believe draw a line almost exactly whore I described 
the emergence of lantjuage -yc tews from a wholly struc- 
turally-rootivate^i approach. 

There is a long way to go y^t, Sot^jo of those whose, 
work I ahrll deacrib^- and whoi^c sytitens i#puld be called 
second generGt:ion, irt- 3tili hli^hlv i^tr ucturalist in 
approach: lu th^.t th. Inferential relations of the struct 
*turcs them. n axcj uhc c jatro of interest • What I 

shall try to uai^ .ain i-N v;e go Al'^^nq that reasonings 
about the world, ray, lr«port^»nt: as they are for under- 
standing langu^vf*^ untiu^t (and tha*^ • ItJ Is now clear 
is rightly i^oen a d^crly w*n pDi5^C53Mion of the Al 
approach to lanru-^g^i, th l'ngn/*ntn having almost altogether 
igrtored it), noncth^f^lecG th^ remains an important level 
" of lingui s tic i nf or ^nco , not r<?dnciblo to reasonings, 
about the world, Let me *gi\*t}i an example here'to txy 
and pin this rather abstract claim down « 

We shall liicaiiVino a number of examples in this course, 

as will others In thoir roursej?, whoro a pronoun in a 

text cannot bo resolve ct (that Is, assigned to its right 

referrent) , unli^^os wo grant some capacity to reason about 

states of affaire, \x\t\ general truthn about people and 

the world. Jo, lor example, in •'The boy next door is 

« 

J «3 
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looking for ^ friend and I hop^ on^ turns up" , we know 

ioinediatly that th^ "one" refers to a friend and not to 

* . , { ~i * , , 

the. boy^ next dooz^J a|id on such a correct assignment the 

If' 

correctness of thW whole translation might rest, I will 

not go into detaiis about tJie rules we might, suggest for 

its resolution, bi^it is clearly not a merely syntactic 

mat^r as it would be^if the rule were simply "assign a 

pronoun to th'^f. last preceding noun", I would suggest 

intuitively that we kWcw'the answer because of^knowl^dge 

th4l| a state of affairs' wani:e<a by one person can be* eV^ 

pected to be one wanted, or not wanted, by another pe^^on, 

and- that a "looking for"f is expected* to be succeeded by 

•a state of "finding" or some other mani f est2Jtion of what 

is sought. This all sounds a little abstract, but it 

could plausibly be ptit in rule form, and then a sequence 

-t * , \, 

of reasonings exhibtliied that would lead us correctly to 

identify "one" with the friend, rather than with the boy, 

Y6u will see a numcer of detailed demonstrations of this 

sort of thing in this and other courses. 

But , and this is\ the point , suppose the sentence had ^ 

been. "The boy next dbt>r is looking for a friend and I hope 

he finds one", then I would say that these reasonings 

might have beQi| a excessive and that an intelligent 

program should get the right answer by simfJly inferring 

t from a repeated pattern equivalent to "PERSON SEEK 

PERSON", by identifying the agents and objects ol^ the 

two occurrences of the pattern , thus yielding the right 

result. This procedure would^Sf course be inferential, 

l^Ut it is not in any simple sense, reasoning about the 

wotid, or at l^st not directly, in the way the earlier 

rea%oni|ig was, ONe of the great problems, as I see it. 
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is how we marry these two modes together ^In a ^ystem that 
knows when to apply the rjight one. I shall return to this 
point in greater detail 4fien discussing my own system. 

The djapox^^t thing tp bear in mind in what follows 
is that "parsing" is bej/ng used in a non-conventional 
wa?: it is not being usjbd in its standard sense in mathe- 
matical and computartiorjfal linguistics, where a structure 
is defined and the method of its application, and the com- 
parison be tween differ/ent methods, is then discussed in- 
^ependfently . That is /what I referred to earlier as the 
"front-end" view of pkrsing, in which it is def ined^^^^_..^^^^ 
narifqwly aind has only) marginal interest. 



In^theseH^o^es "pirsiag" is being used in a. field i-n, 

vhich all proposed structures are up for question: hence, ^ 

tli'ev cannot be assun/ed but must be described in some 

detail first and in/their own terms, as well as in tchrms 

of their adequacy* tp tackle what I called the funcai%^tal 

problems of the systematic ambiguity of natural language. 

. ■ ^ '■'-*» 

Throughout, I shall} take the view, set out above, that the 

possibility of Npar^ing is what gives significance to 
a pitbposed structure (rather than being merely an empil 
cal test of it), in that a good STRUCTURE 6H0WS HCV^ IT 
-~iS TO BE APPLJED — not down to the Very bottom level 
algorithms, but in general form. In these notes, the re- 
fore, you will find a lot of description of the struc- 
tures offered by researchers, and of comparisons between 
thein, and not too much of the actual nltty-g^r itty of the 
sequential forms through which the represeiltation of a 
sentence passes in the actual parsing process. The 
reasons for this fare simple: in the case of some of 
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the best known representatidn systems for Eng|ish, there 
is no effective parsing system as yet, and so to ignore 
those would truncate the Coverage of the course in an 
arbitrary way. Again/ for others, such as Schank's, there 
is a parser in opera^tion, but since Schaftk's system is 
always prteented in a wholly static non-operational way , 
discussions of the structure and its parset are wtfolly^, 
separate, and it is clear which one must be described 
in notes when a competition for space arises as it does 
here. ^ 

however, there will be detailed description of actual, 
blow-by-blow , processing of sentences , based on the 
descriptions and comparisons set out here. These will 
be done by means of transparaiicies , and will include at 
least a display of Winograd*s parsings (concentrating 
particularly on their semantic aspects), and of Riesbeck's 
parser for Sch(ank*s structures, and of my own parser. 
I will also try and introduce some'' details of the actual 
size of the implementations in terms of vocabulary and 
classes of sentences actually processed: a matter it is 
not always easy to elicit from the published literature. 
All I can promise is that I will try to look with 
no more jaundiced an eye at the systems of others than 
I do at my own. 

2. WINOGRAD'S UNDERSTANDING SYSTEM 

Winograd ' s system runs"^>s a dialogue , in real time , 
between a human operator who types in messages, and the 
system propeit, called SHRDLU, which displays on a screen 
pictures of a closed world of coloured blocks and pyramids, 
a box into which they can be put, and an arm or crane that 

I 
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can move the objects about* SHRDLU displays its repl/es 
to the operator by writing on the same screen. The 
following is /a discontinuous sample of the dialoguji 
Winograd giv4s in his book. SHRKLU'S replies are/ in 
upper case : / * - 

pick up the big red block ■ 
OK ,v 

find a block whi?cj:i is taller than the one yo^ are h^^ding 

and put it into ^he box. 

s, 

BY 'IT'^ I ASSUME YOU MEAN THE BLOCK WHICH IS TAL'IjeR THAN 
THE ONE I AM HOLDING 

OK ^ 
what does the box contain 

THE BLUE PYRAMID AND THE BLUE BLOCK ? 

\ 

had you touched any pyramid before you put the 0reen one 
on the little cube J 

YES' THE GREEN ONE 

when did you pick it up 

WHILE I WAS STACKING UP THE RED CUBE' A LARGE RED BLOCK 
AND A LARGE GREEN CUPE.r 

why ^ 

TO GET RID OF IT. 

This ingenious program was also able ta accept new 
definitions of words, such as Sit|e€^j^le ' > to make use 
of what is knew about Ithe properties of the objects 
described, such as that a block could have its top cleared 
off, but a pyramid coul4 not; and also to remember what 
it had done before, as /in the sample above. 

The prograurT^w^Bv^rife in the languagfe PLANNER, which 
is a concrete expression of the slogan "i^eanings are 
procedures" , a sentl/ment into whose own, meaning it is 
probably best not to/ inquire too closely, but which has 

. ' I. • ■ 
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undoubtedly led to a new style ox programming. 



PLANNER 



is a theorem proving/ language: it tr^es to establish the 
ttuth of assertiorts , not in the normal umifdrm, proof- 
theoretic manne^, but by accepting a radge of "programmed 
hints" about hojw best to proceed at anV point. In a 
language understanding program like W:^ograd ' s , this means 
replacing familiar grammar rules sucly as S NP + VP, (a 
sentence consists of a noun phrase £6llowed by a\verb 
phrase) by procedures, in this case/ 

((PDEFINE SENTENCE ((PARSE NP) N II/ FAIL) J ( (PARSE VP)PAIL 



PAIL RETURN) ) ) 



The details of the notatioi/ need not detain us at this 

pointy what is important is tlVat Winograd's grammar is not 

the conventional list of rule^, but small subprograms like 

the lines above, that aetuall^ represent procedures for im - 

posing the desired grammaticlil structnre; 

/ * 
The syntactic structur/e to be imposed is Halliday's 

f 

(1970) hierarchical ^ystem^of clauses, that can be illustrated 



as follows: 



f Clause 



Noun Group 



Verb Group 



Noun Group 



Number Adjective Noun / Verb Adverb Verb Adj Adj 



Noun 



Noun 

I 



TwD foolish children were avidly gobbling hot fresh eggplant pancakes 



> 



$ Actually the grammar was written In a language PROGRAMMpj^, 
and the semantic and inference parts of the program in LISP 
and a subset of PLANNER called MICRO-PLANNER. ' 
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In terms of th6 notions setUut earlier Winograd's parsing 
is top down, and depth first, with no automatic back up. 
The parsing program ^or each gLramm*tical category is a fun*- 
ctional definition in PROGRAMMAR, which 9an be stated either 



as above for SENTENCE, or as a flbiir-chart as below for VP: 



DEFINE program VP 





Here is Winograd'3 own account of the start of this 
top-down parsing procedure for the sentence "Pick up a 

big red block" « 

"The CLAUSE program looks at the first word, to decide 
what unit the CLAUSE begins with^ If it sees an adverb, it 
assumes the sentence begins with a single-word modifier^ 
if it sees a preposition, it looks for an initial PREPG, 
If it sees a BINDER, it calls the CLAUSE picogram to look 
for a BOUND CLAUSE- In English (and possib^ly all languages) 
the first word of a conitructiorj^ often gives a very good 
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clxxe as to what that construction bill be. In this case, 

pick'' is a^'verb, and indicates that we may have an IMPERATIVE 
CLAUS&i^ The program Istarts the VG program with th.e initial 
VG feature list (VG IMPER) , looking for a VG of this type • 
^his must either begin with some form of the verb "do'% or 
with the main verb itself • Since the next word is notV'do" 
it checks "the next word in the input (in this case still 
the' first word) to see whether it is the infinitive form 
of a verb. If so, it* is to be attached to the parsing 
tree# and given the additional feature nVB (main verb). 
The current structure can be diagrammed as: 

(CLAUSfi^^^AJOR) 
(VG IMPER) 

(VB MVB IN^^i^ANS VPRT ' pick 

TRANS AND VPRT came from the definition of the word ^pick" 
when we called the function PARSE for a word." 

After the syntactic parsing, a number of "semantj^ 
specialists" attach semantic structures to Specific syntactic 
ones. A semantic de finition of an action like "grasp" would 

be of the form * 

(CMFANS { { { ( ^-ANIMATE) ) { { M MANIP) ) ) 

t 

{ #^EVAZ (COND ( (PROGRESSIVE)^ 
(QUOTE ( *j^GRASPING ^2 *TIME) ) ) 
(T (QUOTE ( #-GRASP i??^2 *TIME 

) ) ) ) ) NIL^) ) 

\ 

which says essentially that grasping is something done by 
an animate entity to a roanipulable one (first line) . More 
of the real content of such actions is found in their 
inferential defihition. Her^ is the one for "pickup**: 
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(DEFTHEOREM TC-PICKUP(THCONSE (X (WHY— lEV) ) EV) 

("PICKUP $?X) (MEMORY) (THGOAL^t^RASP $?X) (THIJSE TC-GRASP)) 
(THGOAL (*IRAISE«AND) (THNODB) ^^^THUSE TC-RAISEHANp) ) ^ 



(MEMOREND (JiPICKUP $?EV $?X) ) ) ) ^ 
Orvce again th^ details of the notation ncdd not be ex- 
plained in order to see t^at the word is being defined in 
terms of a number of more primitive sub-actions , such as 
RAISEHAND* each of wViich must be carried out^ln order^^^>»t 
something may indeed be picked up. 

In the case of "a red cube", the following structure 
is built up by an NP "semantic specialist" 
(GOAL (IS X BLOCK) ) 
•(GOAL (COLOR X RED)) 

(EQDIM X ^ PLANNER dGScrlption 

(BLOCK MANIP PHYSOP THING) markers 

The first three lines are procedures that when evaluated 
will seek an object X that is a block, is eq'uidimensional 

"'1 
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(EQDim) and is red. The last ^ine is a set of "semantic 
features** rea;i off fight to left from the following ''feature 
tree** 

< NAME ^ , 
• » PLACE I SHAPE 

» PROPERTY jSIZE 



I ANIMATE- 



! LOCATION 
(COLOR 



I ROBOT 



THING — - 

"v ^ 



I 

i PHYSOB— 



I HUMAN 

I BLUE 
I RED 

I BLACK 

'iWHITE 

IGREE^N I STACK 

ICONSTRUCT- * IPILB 

(HAND I ROW 

— ITABLE IPYRAMID 

^ iMANIP jBLOCK 

I BALL 



RELATION- 



I EVENT 

i 



(TIMELESS 

This whole semantic structure can be used by the deductive 
component at the system, before evaluation resulting in the 
actual picking up, to see if such an object is p^s5^ij>le. 
It it were not, (an *' equidimensional pyramid" would not be), 
the system could go back and try to re-parse the sentence. 

One reason for the enormous impact of this work was that, 
prior to it^s appearance, AI work was linguistically trivial, 
while the systems of the linguists had ifHo place for the use 
of inference and real world knowledge. Thus a very limited 
union between the two techniques was able to breed consider- 
able results. Before Ulnograd there were few programs in 
AI that could take a reasonable complex English sentence 
and ascribe any structure whatever to it. In early classics 
of 'natural language understanding* in AI, such as Bobrow's 
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STUDENT (1968) problem|tsolver |for simple algebra, input 

i, I 

sentences had to be shbrt and bf stereotyped form, \ 



such as ''what is the s\^m of 

Conversely, in linguistics, 
cently, little speCulatlion on 



there was , until very re- 
ow we understand the re- 



ference of promouns in such elementary sentences **t^e 
soldiers* fired at the wAmen and\ I saW several fall** # 
where it is clear that tme an s war is both definite , and 
that finding it requires! some inferential manipulation o|f 
generalizations about the world.! The reader should ask \ 
himself at this point how, he knows the referent ox the \ 
pronoun in that sentence. | 



SOME DISCUSSION OF SHRD 



So far, the reaction to iwinograd ' s work has been wholly 
uocritiqal. What would critics fi\\A to attack if they were 
saifiiinded? Firstly, that Wihograd's linguistic system is 
highly conservative, and thai the distinction between 'syn- 
tax and 'semantics* may not be necessary at all. Secondly^ 
that his semantics is tied to the simple referential world 
of the blocks in a way that would make it inextensible to ; 
any general, real world, situatibn. Suppose 'block' were 
allowed to mean 'an obstruction' and 'a mental inhibition', 
as well as *a cubic object'. It is doubtful whether Hinograd's 
features and rules could express the ambiguity, and, more 
importantly, whether the simple /structures he manipulated 
could decide correctly between the alternativf* meanings 
in any given context of use. Again, far more sophisticated 
and systematic case structure^' than those he used might be 
needed to resolve the ambiguity of 'in' in 'He ran the mile 
in five minutes', and 'He- ran the mile in a paper baCj', as 
well as the combination of c/ise with word sense ambiguity 
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in 'He put the key in the lock* (door lock) and • He threw 
the key in the lock** (fiver lock) . 

The blocks world is also strongly deductive and logi'cally 
clo.sed* If gravity were introduced into it, then anything 
supported that was pushed in a certain way would have, 
logical ly have # to f all • But the * common sense world , of^ 
ordinary language, is not like that: in the 'women and 
soldiers • exacjple given earlier , the pronoun • several • ca^ 
be said to bo resolved using some generalisation such as / 
•things shot at and hurt tend to fall'. There are noLo/gical 
•have to • 8 • there, even though the meaning of the prQiiou|^is 
perfectly definite* ^ "'^^^^^..^ \ 

Indeed, it might be\argueaj that, in a sense ,\ and as 
regards its semantics , Winograd's system is not^uabout 
natujral language at all, but aboat the other tecRfnlcal 
question of how goals and sub~goals are. to be organised in 
a problem-solving system capable of manipulating simple 
physical objects. If one. glances back at the def ini tion * of 
•pickup^ quoted above, one can see that it is in fact an ^ 
expression of a procedure for picking up an object In the 
SHRDLU system, Nothing about Itr for example, would 
help one understand the perfectly ordinary sentence •l 
^picked up my bags from the platform and ran for the train 
let alone any sentence not about a physical action per- 
formable by the hearer. Gnu* could put the point so: what 
we are given in the PLANNER code is not a sense of "pick 
up* but a case of its use, just as 'John picked up the 
volunteer from the audience by loaning over the edge of 
the stage and drawing her up by moans of a ^oi^ clenched 
in his teeth*' is not so much a sense of the verb as^ a 
use of it. 
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Those who like very general analogies may have n<fticed 
that Wittgenstein (1953. para. 2ff. ) devoted considerable 
space to the construction of an elementary language of 
blocks, beams and slabsj o^e postulated on the assumption 
that the words of language were basically, as is supposed 
in model theory, the nam^s of items. But, as- he showed 
Of the enterprise, and t^ the satisfaction of many readers, 
•*That philosophical concerpt of meaning (i.e. of words as 
the unambiguous names of physical objects — -^YW) has its 
place In a primitive idea of the way language functions. 
But one can also say that it is the idea of a language 
.more primitive than ours ". (my italics). Thus in terms 
of the notions of the introduction onaf might say that there 
is a strong structurally (rather than content) motivated 
element in SHRDLU. 

In this cqnrsQ I do not discuss tlie excellent and 
widely )cnown work of Woods. Let me explain why. 
The system, based on an augmented state transition net- 
work grammar, is undoubtedly one of the most robust in 
actual use, in that it is less sensitive to the PARTICULAR 
input questions it encounters than its rivals. The reason 
for not treating it in depth is that both Woods and Wino- 
grad have argued in print that their two systems are 
essentia lly equivalent (Winogra<J- 1971) (Woods 1973), and 
so, if they are right, therQ-is no need to discuss both, 
and Winograd's is, within^he AI community at least, the 
better known of the <twc. 

Their equivalence arguments are probably correct: both 
are grammar-based deductive systems, operating within a 
question-answering environment in a highly limited domain of 
discourse. Winograd's system of hints on how to proceed. 
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within his PROGRAMMAR grammar, is, as he himself point out, 
formally equivalent to an augmented stato transition net-- 
work, and in particular to the ordering of choices at nod^s 
in Woods' systeldQ. 

There is a cignificaflt difference in their metaphysical 
approaches , or presuppositions about meaning which , however , 
has no influfence on the actual opetation of their respective 
systems. This difference is disguised by the alliegiance 
both give to a 'procedural view of meaning*. The difference 
is that Woods takes a much more logico-semantic interpretation 
^of that slogan than doos Winograd. In particular, for Woods 
the meaning of an input utterance to his system is the pro- 
cedures within the system that manipulate the truth conditions 
of the utterance and establish its truth value. To put the 
matter crudely, fbr Woods' an assertion has no meaning if his 
system cannot establish its truth^or falsity. Winograd has 
certainly not committed himself to any such extreme position. 

It is interesting^ to notice that Woods' is, in virtue of 
his strong position on truth conditions, probably the only 
piece of work in the field of AI and natural language to. 
satisfy Hayes' (1974) recent demand that to bo "intellectually 
respectable" a knowdedg^j system must have natural model 
theoretic semantics, in Tarski's sense* Sirfce no one has 
ever given precise truth conditions for any interesting 
piece of discourse , such as, say. Woods * own papers , 
one might claim that his theoretical preswrpposi tions 
necessarily limit his work to the analysis of micro-worlds 
(as distinct from everyday languaqe)^ 

There is a low- love 1 problem abof t the oqui valance of 
Woods' and Winograd* s systems, if we consider what we might 
call thfe received common-sensc view of their work. Consider 
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the following three assertions: ^ ^ 

(1) Woods' system is an implementation of a 
transformational grammar* 

(2) Winograd's work has shown the irrelevance 
of transformational grammar for language 
analysis-^a view widely held by reviews of 
his work • , ^ ^ " 

<3) Woods' and Hinograd's systems are formally 
equi valent--a view held by both of them. 
Thei;e is clearly something of an inconsistent triad 
amongst those three widely held beliefs. The trouble 
probably centers on the «xact sense which i^^oods • work is 
formally equivalent to. a transformational grammar--not a 
question that need detain us here, but one worth pointing 
out in passing. 

4. SECOND GENERATION SYSTEMS 

'To understand what w^s meant v*hen Winograd contra«1:ed, ' 
his own with what ho catl^ed second generation systems, we 
have to remember, as always in this subject, that the gen- 
erations are of fashion, not chronology or inheritance of 
ideas. He described the work ot Simmons, Schank and myself 
among others in his survey of new approaches, even though 
the foundations and terminology of those approaches were 
set out in print in 1966, 1^*07 and 1968 respectively. v;hat 
those approaches, and others , have in common is the belief 
that understanding sysj^cms muat be ablo to manipulate very 
complo^c Ittiguistic cbjfectLj, or semantic structures, and 
that no simplistic approachoti to unduratanding language 
with computers will work . 
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m -an unpublished, but already very influential n^gceitt 
p^per, Minsky {TC^y/ has drawn together strands in J^e work 
of Charniak Jl^;^^and the authors above using a terminology 
d¥ 'frames'.: 

"A frame is a data- structure for representing a-s^Sxio- 
type situation, like a certain kind of living. r<^m, or going 
to a children-? birthday party. Attached to eacl* frame_jire 
several kinds of information. Some of this is iinf ormCHon^,^ 
about how to use the frame. Some is about what one can exz.^ 
p^ct to happen next. Some is about what to do if these ex- 
pectations are not confirmed. 

^ We can^think of a frame as a network of nodes and rela- 
tions. The top levels ^fj/ frame are fixed and represent 
things that are always true about the suppose^ situation. 
The lower levels 'have many terminals-— 'slots' that ^att»t 
be filled by specific instances or date. Each terminal can 

specify conditions its assignments must meet Simple 

conditions are specified by markers that might require a 

terminal assignment to Ewe a person, *an object of suffictenl 

value, etc...." " 

The key point about such structures is that they attempt 
AO specify in advance what is going to be said, and how the 
world encountered is going to be structured. The structures, 
and the inference rules that apply to them, are also ex- 
.^^p^ssions of • partial inf ormatior>' (in McCarthy's phrase) 
that are not present in first generation systems. As I 
showed earlier, with the •wom^n and soldiers' example, s^ch 
loose inductive information, seeking confirmation from the 
surrounding context , is required for very simple sentences. 
lA psychological and vis^aal terms, frame approaches Envisage 
an understander as ^at least as much a looker as a seer. 
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I shall now dertribe briefly three approaches that might 
be called second generation > Colby, Schank and myself. I 
\^ shall not*dLfecuss Charniak's work here, largely because he 
\'85<iescribi4s it himself # in another course , but partly be- 
ctfliseHh^e does not implement his system with a parser. He , 
also^i^iSteigrees with th** view of parsing I sketched in Section 
I — nal^£v that a representational structure gets much of 
# its signlftkcS^nce from its actual ""^boj^ji^otential parsing, and 
'•^^hat* parsing ^er«Lf ore^^S^nnot be decoupled — I shall return 
tor ht^, work in t^i^\^a^t1^r comparisons and assume some know- 
. ledger^ of ^^^t^ by my audience . 

Colby 

Colby ha^ (Colby et al '{11), (Enea a|jd Colby '73) , 
(Colby and Parki^on, •74) d^velcf|jfed over a period of sever^ti.* 
years an on-line (dialogue system that: simulates a paranoid 
patient o^H^ PARRYv A ineas"t3nre of its succei&s is that it 
is perhaps the &q3t"us^dj^I program in existence^ used, 

thkt is, by those b^utsid^ the research^ pro ject that developed 

\ ■-■ ^ 
Here is a d£>6corltinuons sample of its output, again 

ith the s^ys ten responses In the upper case: 



_JJavR "ou been hospi ta'lbi^ed before? 
^ THIS IS THE FIRST TIM§>^_J^ _ 
How i^ng have 2 0^i be^n^h^re? ^ 



ABOUT TItfO WEEKS. 



Afty headache*? \ < i^^ 
M^ afiALTH IS F*I^E. 

"Are\you havijig any difficulty with your memory? 
NO. / ^ 

Then perhaps you* can^^'^^Ll itoe who ^s the President of the 
United States? 0" : v 

NIXQN*^ IS PRESIDENT, 
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What day is 'it today?^ 
TODAY IS THURSDAY. 
What season Is it? 

LETS GET ON WITH THE INTERVIEW.- 

Colby's aid was to produce a system that was not, in 
his word", 'fragile' in the way that sosi: Ai systems are: 
one th^t did not break down, or give up, if the input was 
not exactly what it expeet^d; but one that carried on some- 
how-^n all circumatancek , i>i the way that conversing humans 



\ 



do* y 

* Fragility^ is aVoT^d/bry having no conventional s/fctax 
analyser, and by not erven attempting to take account of all 
the words in the input. This is a considerable aid, since 
any parser that begins\to parse a !5ore than usually polite 
requj^ such as 'Would yob. be so kind as to...' is going 
to in trouble. British \:nglish speakers arriving in 
the U»S.*» quickly learn to dtelete such phrases, since they 
cause great confusion to human listeners in stores. 

The input text is segmented by a heuristic that breaks 
it at any occurrence of a ran^ of key w^rds. Patterns are 
then matched with each segment. ^There are at present about 
1700 patterns on a list. (Colby aid Parkison, in press) that 
is stored and matched, not against any syntactic or semantic 
representations .of words (excepjjt to deal with contractions 
and misspelling^T', but against the inpiHT^ord string direct, 
and by a process of sequentjUjL de^te^^n. So, for example, 
•What is your main problem'Sias a root verb 'BE' substituted 
to become 

. WHAT BE YOU MAIN PROBLEM. 
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It is then matched successively ir^ the following forms after 
successive deletions! 



BE YOU MAIN PROBLEM | 
WHAT YOU MAIN PROBLEM 
WHAT BE MAIN PROBLEM 
WHAT BE YOU PROBLEM 
WHAT BE YOU MAIN 

and only the penultimate line exists as one of the stored, 
patterns and so is matched. Stored in the same format ias 
the patterns are rules expressing the consequences for the 
' patient • of detecting aggress ion and over-- friendliness in 
the interviewer's questions and remarks* The matched pat- 
terns round are then tied directly, or via these inference 
rules, to response patterns which are generated. 

Enormous ingenuity has gone into the heuristics of 
this system, as ^ts popularity testifies. The system has 
'also changed considerably: it is now called PARRY 2 and 
contains the above pattern matching, rather than earlier V 
key work, heuristics. It has the partial, or what some 
would call 'pragmatic' , rules about expectation and inten-- 
tion, and the<»e alone might qualify it as 'second generation' 
on some interpretations of the phrase. A generator is also^ 
being installed to avoid the production of only 'canned' 
responses . 

Colby and his associates have put considerable energy 
into actually trying to find out whether or not psychiatrists 
can distinguish PARRY'S responses from those of a patient 
(Colby and Hilf '73). This is probably the first attempt 
to actuarily apply Turing's test of machine-person distin- 
guishabijity . There are statistical dif f icfulties about 
interpreting the results but, by and large, the result is 
that the sample questioned cannot distinguish the two. 
Whether or not this will influence those who stilj^ on prin- 
ciple, believe that PARRY is not a simulation because it 



does not understand' f remains to be seen* It might be 
argued that they are In danger of falling Into a form of 
Paperl's 'hi 



luman-superhumim fallacy': of attacking machine 
simulations because th'ey do not perform superhuman tasks , 



J like translate poetry, tasks that some people certainly 
cant do. When such sceptics say that PARRY does not 
stapd they, have In their minds a level of understanding that 



ider- 



is jdertalnly high - one could extend their case ironically 
by pointing out that very few people understand the content « 
of sentences In the depth and detail that an analytic philo- 
sopher does, and a very good thing too. There can be no 

doujbt that many people on many occasions DO seem to understand 

i 

In ,;the way that PARRY does. 
Schank 



% Is a rich system of semantic representation , 



Schai 

developed over a period of six years, with the collaboration 
of a npmber of talented students. Its graph sys^tem of 
notatJ/on has Influenced psychologists like Abelson (1973) , 
amono^ o ther 8 . Schank 's contribution has been the notatlonal 
system representing the s'tructure of natural language sen- 
tences, and this has been programmed by various collaborators 
over the years. In Its present version, called MARGIE 
(Schank et al '73) It has an analyser of English due to 
Rlesbeck (1974) , a semantic memory component due to Rleger 
(1974) , and a generator of English due to Goldman (1974) . 
MARGIE produces output In ^wo modes, ^^^onstratlng the sort 
of conceptual Inferenclng that^oes on at the level of the 
semantic^ representation : the PARAPHRASE mode and the INFE- 
RENCE mode. Samples of Input and output to and from the 
'^wo modes can be shown thus: / 
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$ (Footnote on page 31) 
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INPtJT: John told Mary that Bill wants a book. 

OUTPUT? A book about what 

Mary knows that Bill wants a book. 

Bill wants to come to have a book ♦ 

Bill wants someone to cease to have a book. 

(PARAPHRASE MODE) 

INPUT: John killed Mary by choking Mary. 
OUTPUT: John strangled Mary. 

I 

John choked Mary and she died because sh^ could not breathe. 

Mary died because she was unable to inhale some air and she 
^was unable to inhale some air because John grabbed her neck. 

The a^ of Schauk's system hds always been to provide 
a rcpresent2|tion of meaning, in terms of which these and other 
tasks, such as machine translation, could be carried out; 
a representation, moreover, that was to be independent of 
any particular languages, and of syntax, and indeed, of all 
surface structure whatever. 

I Tlie formal structure^ of Schank's graphs is that of de- 
pendency gramw»*r (Hays '64), and the items in the graph ate 
of four types, or conceptual categories. They are symbolised 
as PP, ACT, PA and AA , which are acronyms, but which y\ 
correspond closely (for the purpose of understanding tneir 



$ Schank dis tirtguishes 'conceptual' and 'semantic' reprcsen-- 
tations in a way that is important for him within hi« ov;n 
system. However, I shall use the terms indifferently since, 
in ^This brief and superficial description, nothing hangs 
upon the distinction. 
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function ) to those of a n^rrT verb , adjective and adverb , 

respectively* The hp^Cc structure Is called a conceptual Is a - 

tlon 9 and Is norjriTlly Introduced with a straightforward de- 

pendency str)>^ture such as r for the sentence 'The man took a 

book": y ^ 
♦ 

*^ . p o 

Man^~-^take < book 

Here 'p' Indicates past, arid>i.^t^h^ dependency symbol linking 
a PP td , the ACT ('take') which Is thV^tmh^of the conceptual- 
Isatlon^ as with Simmons. The 'o' Indicates the objective 
case^ marking the dependence of t-he object PP on the central 
ACT. There Is a carefully constructed syntax of linkages 
between the conceptual categories, that wll^ be described 
only In part In what follows: 

The next stage of the notation Involves an extended casd 
notation and a set of primitive ACTS, as well as a number of 
It^ms such as PHYCCONT whlc^ Indicate other states , and Items 
of a fairly simplified psychological theory (the dictionary 
entry for advise, for example, contains a subgraph telling 
us that y ' will benefit' as part of the meaning of 'X 
advises y (Schank '73)). There are four cases in the 
system, and their subgraphs are as follows: 



^ Objective ckse: hCT ^ PP 



iPP 

Recipient case+ ACTf- 

:pp 



■HZ 

» CI 



♦ This is a considerable oversimpli flea tlon , made in order 
to give a brief and self contained description . But , in fact , 
many English nouns are representjed as ACT ' s : chair , pen , 
Ijl^nesty, and transportation. 
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Instrumental case: ACT 

->PP 



Directive case: ACT 



-<PP 

/• 

There are at present fourteen* Ibasic actions forming the 
nubs of the graphs, as well*^as a default action DO. They 
^re:> PROPEL} MOVE J INGEST J EXPEL J GRASP J PTRANSJ MTRANS , 
ATRANS, SMELL, SPEAK, LOOK-AT, LISTEN-TO, CONC and MBUIL%.. 
The notions of case and primitive act^are related by rules 
in the development of conceptualisations. So, for example, 
the primitive act INGEST has as its instrument the act 
PTRANS. There are also other inferences from any ACT classi- 
fied as an IMGEST action, such as that the thing ingested 
cl^anges its form> that if the thing ingested is edible the 
ingester becomes • more nourishod* etc. C'see Schank *13, 
pp. 38ff.). This will all become clearer if we consider 
the transition from a dict^nary entry for an action to a 
filled-in conceptualisation. Here is the dictionary entry 
for the action 'shoot': 

o p 
X ^ — ^ \ PROPEL f "^bullet ^ 



t 



' — <gun 



Y <C > s hurt 

VJe can consider this entry as an active 'frame-like' object 
seeking filler items in any context in which it is activated. 
Thus, is the sentence 'John shot the girl with a riflo', the 
variables will be filled in from context and the case in- 
ference will be made from the main act PROPEL, which is that 
its instrument is MOVE' GRASP or PROPEL, and so we will 
arrive at the whole conceptualisation : 

* Since the publication of (Schank 73a) their number has been 
reduced to eleven (plus DO) by th*^ elimination of SMELL* 
LISTENTO' LOOKAT and GONG, and the addition of ATTEND. 
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rifle 

— ^giri 

Johnx^=a=>PROPBL 

bullet ^ 

- N PHYSC0NT 

girl 

bullet 

r^^ — ^ 

rifle girl 

This case inference must \>e made, according to Schank , in 
order to achieve an adequate representation. There is, in 
the last diagram, a certain redundancy of expression, but 
^^^a» we shall see in the next section this often happens with 
deeper semantic notations. 

More recently , Schank , together wi th Rieger , ha^ devel* 
oped a new class of causal inferences which deepen the 
diagrams still further. So, in the analysis of 'John's cold 
Improved because I gave him an apple* (in Schank ^^74a) the 
extended diagram contains at least four yet lower levels 
of causal arrowing, including one corresponding to the no- 
tion of John constructing the idea (MBUILD) thkt he wants 
to eat an apple. 'So we can see that the underlying explica- 
tion of meaning here hm^ not. only in the sense of linguistic 
primitives, but in terms of a theory of mental acts as well. 

NOW there are a number of genuine oxpositional difficul- 
ties hero for the commentator faced with a system of this 
complexity . One aspect of this is the stages of development 
of the system itself, which can be seen as a consistent 
process of producing what was argued for in advance. For 
example, Schank claimed early on to be a constructing 
system of semantic structures underlying the 'surface of 
natural language ' , although initially there were no primi- 
tives at all, and as late as (Schank et al '70) there was 
only a single primitive TRANS, and most of the entries in 
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the dictionary consisted of the English words coded, together 
with subscripts. Since then the primitive system has blossomed 
and there are now twelve primitives for ACTS including three for 
the original TRANS itself- Each exposition of the system re- 
counts its preceding phases, from the original primitive- 
free one, through to the present causal infer^eAce form; 
rather as each human foetus is said to relive in the womb 
all tjie evolutionary stages of the human race. The only 
trouble with this, from an outsider's point of view, is that 
at each stage the representation has been claimed, in firm 
tones, €o ^e the correct one, while" at the same time Schank 
admits, in moments of candor (Schank '73), that there is 
no end to the conceptual diagramming of a sentence. This 
difficulty may well reflect genuine problems in language 
itself, and, in its acutest form concerns a three-w^ay con- 
fusion between an attractive notation for displaying the 
'meanings of words', the course of events in the real 
world', and, finally, actual procedures for analysis. It 
is not always clear wh'?ther or not proocedures implementing 
conceptual depondenca are intended to recapture all the ^ 
many phases of expansion of the diagrams. 

This raises the, to roc, importarj^t question of the ap- 
plication of a semantic system, that I shall touch on again 
later* Schank, for example, does mention in passing the 
questions of wo*rd-sense ambiguity, and the awful ambiguity 
of English prepos}.t ions , but they are in no way central for 
him, and ho assumes that** with the availability of 'the cor- 
rect representation', his system when implemented must in- 
evitably solve these traditional and vexing questions. No 
procedures are hi^ed at along with the graphs as to how this 
is to be done. A distinction of importance may be becoming 
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apparent here between Schank's work and Rleger's: in Rieger's 
thesis (Rieger '74) the rules of i^nference appear to create 
separate and new subgraphs which may stand in an inferential 
relation to each other so as to produce conclusions about 
problems of, say# pronoun reference, etc. But in Schank * s 
corresponding papers the same inferences are not applied to 
actual problems (Schank '74a) ttut only complexify the con- 
ceptual graph|i further. Closely ^Connected with this is the 
question of the survival of the surface structure in the dia- 
grams. Until' very recently primitiveisation applied bnly to 
verbs# that of nouns being left to Weber (Weber '72). Most 
recently, though, itoun word» have boon disajppearing from ^ 
^ diagrams and been replaced by categories such as 'PHYSOBS*. 
But it is clear that the sujf^face is only slowly disappearing, 
rather than having been abhorred all along. 

In his most recent publication (Schank '74b) there are 
signs that this trend of infinitely proliferating dia'grams 
is reversing. In it Schiink coasiders the/ application of 
his approach to the representation of t^xt, and concludes, 
correctly in ran view, that, the representations* of plSirts 
of the text -must be interconnected by causal arrows, and that 
in order to preserve lucidity, the conceptual diagrams for 
individual sentences and their patts must be abbreviated, 
as by triples such as PEOPLE PTltAltS PEOPLE- Here indeed, the 
surface simply has to survive in the representation unless 
one is prepared to commit oneself to the extreme view that 
the ordering of sentences^ in a text is a purfe*ly superficial 
and^ arbitrary matter . The sense in wliich this is a welcome 
reversal of a trend should be clear , because in the ' causa-- 
tioh^^nf erence * development , mentioned earlier , all the con- 
sequences and effects of a conceptualization had to be drawn . 
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within itself*: 3«hu8, in the extreme /case . each sentence 
of a text should have been represented by a diagram containing 
most or all of the text of which Xt wa* a paft. Thus the 
representation of a text would have been imiossible on such 
principles. ■ J 

At this point I shall describe in sdi^e det^iTl' with 
slides the implementation of Scbank's system as a paxser 
by Rlesbeck. / • 



Wilks 



My own »y«te| also has a uniform representation . in terms 
of structures of primitives, for' the 6ontent of natural 
language. It is uniform in that information that might 
conventionally be considered syntactic, semantic or factual 
is Jill represented within a single structure of complex 
entities* all of which are in turn constructed from a budget 
of 80 primitive semantic entities. 
J The system runs on-line as a package of LISP and MLISp 
programs, taking as input small paragraphs'* of English, that 
cart be made up by the user from a vocabulary" of about 600 
words, and producing a good French translation as output. 
This environment provides a pretty clear test of language 
understanding, because French translations for everyday 
prose are either right or wrong, and can b€ seen to be So. 
While at the ^ame time, the major difficulties of understanding 
programs word sense ambiguity, case ambiguity, diffi- 
cult pronoun reference, etc. , can all be represented 

within a machine translation onviroriroent by. for example, 
choosing the wards of the Input sontetice containing a pro- 
noun reference difficulty that the possible alternative 
references have different genders In French. m that way 
the Fr^rioh output makes quite clear whether or not the 
♦called formulas and paraplates ~ 
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program has made the correct inferences *in order to lyvder- 
stand what it is translating* The program is reasonably 
robust in ^ctual performance, and will even tolerate a certain 
amount of bad grammar in the input, since it is not performing 
a syntax analysis ih the conventional sense, but seeing 
messages representable in the s^|jp^tic structures employed. 

Typical input would be a sentence such as "John livea 
out of town, and drinks, his wine C3?ut of a bottle. He €hen 
throws the bottles out of the window." The program will 
produce French sentences with different* output for ekch of 
the three occurre^nces of "out of's'^ince it realises that 
they function (Juite differently on the three occasions of 
use, and that the difference must be reflected in the French^ 
A sentence such as "Give the monkeys bananas although they 
are npt ^ipe because they are very hungry" produces .a trans- 
lati^n with different equivalents for the two occurrences 
of ^they", because the system correctly realises, from whit, 
1 shall describe below as preference considerations, that 
the most sensible interpretation is one in which the first 
"they" refers to the: bananas and the secdnd to the monkeys, 
and bananas and monkeys have different genders in French. 
These two examples ar^ dealt with in the 'basic »Ode" of 
the system (Wil)cs ' ' 73a) . In many cases it cannot re^iolve 
pronoun ambiguities by the sort of straightforward "prefer- 
ence considerations" used in the last example, where, roughly 
speaking, "ripeness" prefoirs to be predicated of plant-like 
things, and hunger of animate things. Even in a sentence 
as simple as "John drank the wine on the table and it was 
good", such considerations aro inadequate to resolve the 
ambiguity of "it" between wine and table, ainco both may be 
good things. In. such cas^^s, of inability to resolve within 



3,'} 



39- 

its basic mode, the program deepeas the representation of 

the text so as to try and set up chains of inference that 

will reach, and so prefer, only one of the possible referents. 

I will return to th^se processes in a moment, but first I i 

shaLl give some brief description of the basic representation* ^ 

set up for ♦English. ^ ^ 

Fpr each sense of a word in its dictior\^ry the^ program 
sees *a formula . This is a tree structure of semantic primi- 
tives, and is to be interpreted fprmally u^ing de'pendency ^ 
relations. The main element ip any formula is the rightmost, 
called its head, and that is the fundamental category to 
which the formula belongs. In the formulas for/ actions, for 
example, the head will always be one of the primitives 
PICK, CAUSE, CHANCE, FEEL, HAVE, PLEASE. PAIIi, SENSE, USE, . 
WANT, TELL, BE, DO, FORCE, MOVE, WRAP, THlNIC^ FLOW, MAKE , ^ , ^ 

DROP, STRIKj FUNC or HAFM. 

Here is the tree 84:riicturc for the action of drinking^ 




(THRU PART) 



Onc^ again, It is not nocegsary to explain the formalism 
in any detail, to see that this sense of "drink" is being 
expressed as a causing to mov^^ a liquid object (FLOU STUFF) 
by an animate agent, into that same a'^ent (containment case 
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indicated by IN, and formula syntax inditifies SELF with the 
agent) and via (direction case) an aperture (THRU PART) of the 

agent.' O 

Template structures, which actually repre^nt sentences and 
their paifts are built up as networ)cs of formulas like the 
one above Teinplates always consist of an agent node, and 
action node and an object node, and other nodes that may de- 
fend on these. So, in building a template for -John drinks- 
wine", the whole of the above tree-formula for "drinks" 
would be placed at the Action node, another tree structure 
for "John' at the agent node and so on. The complexity of 
the system comes from the way in which the formula^, cbn- 
sidered as active entities, dictate how oth^r plaices in 
the same .template should be filled. 

Thus# the "drink", formula above can be thought of 
an entity at a template action node, seeking a J.iquid 
object, that is, to saw a formula with (FLOW STUFF) as its 
right-most branch, to put at the object node of the same 
template. This seeking is preferential , in that formulas 
not satisfying that requirement will be accepted, but only 
if ^nothing satisfactory can be foujid. The templage finally 
established for a fragment of text is the one in which the 
most formulas hav^ their preferences satisfied. There is 
a g^eneral princip/e at work here: that the right interpre- 
tation "says the least" in information-carrying terms . 
This very simple device is able to doiAiuch of the work ^ 
of a syntax and word-sense ambiguity resolvvn^LProgram. 
For example, if the sentence had been "John dranl^ a whole 
Pitcher", the formula for the "pitcher of liquid" would 
Have been preferred to tfffft^ f or the human, since the 
subformula (FLOW STUFF) could be appropriately located 
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within it. 

A considerable amount of sq^j^eezing of this simple canoni- 
cal form of t^plate is necessary to make it fit the complex 
ity of language: texts have to be fragmented initially; 
then, in fragments which ar^, say, prepositional phrases 
there is a dummy agent imposed, and the prepositional formu- 
la fiAjnr^tions as a pseudo-action. There are special "l^ss 
prefe-rred" orders to deal with fragments not in agent-action 
object order, and so on. 

When ^^^l^^^^a.l inferences have been done that set up the 
agent-action«object templates for fragments of input text, 
the. system^ attempts to tie these templates together so as 
to provide an overall initial structure for the input. One 
.form of this is ^the anapii^ra tie, of the sort discussed 
for the "monkeys, and bananas** example above, buV^he more ^ 
general form is the case tie. Assignment of these would 
result in the template for the last clause of **He ran the 
mile in a paper bag** beij>g tied to the action node of the 
template for. the first clause **He ran the mile**, and the tie 
being labelled CONTainment. These case ties are made with 
the aid of e^nother class of ordered structures called 
parapl^tes , that are attached to the formulas for English 
prepositions. So, for **outof ** there would be at least six 
ordered paraplates, each of which is a string of functions 
that seek inside templates for information. In general, 
paraplates range across two, not necessarily contiguous, 
templates. So, in analysing "He put the number he thought 
of in the table^*, the successfully matching paraplate would 
pin down the dependence of the template for the last of the 
three clauses ad DIREction, by taking as argument only that 
one template for the last clause that Contaij^ed the formul^i 
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for a numerical table, rather tftan a kitchen table, in 
virtue, in this example,. of the function in that paraplate 
seeking a similarity of head tsiGN in this case) between th 
two object formulas, for "number" and "table". 

The structure -of mutually connected templates that has 
been put together thus far constitutes a "semantic block", 
and if it can be constructed, then, as far as the system iH 
concerned, all semantic ahd referential ambiguity has been 
reVolved and it will begin to generate Franch by unwrapping 

l3^ck again. The generation aspects of this work have 
been^deVci^i^e'd in {iae^rskovi ts ) . One aspect of the gene- 
ral notion of pr^^rence is that the system should never 
^bnstriic^t a de^er^or morfi. elaborate semantic representation 
than is nepessary fcTr the task in hand and, if the initial 
block can be constructed and a generation of French done, 
no "Deepening" of the representation will be attempted. 

However; many examples cannot be resolved by the methods 
of this "basic mode" and, in particular, if a word sense 
ambiguity, or proni^un reference, is still unresolved, then 
a Uniqoie semantic block of templates cannot be constructed 
and the "extended mode" will be entered.* In this mode, 
new template-'like forms are extracted from existing ones, / 
and then added to the template pool from which further iry^ 
ferences can be made. So, in the template derived earlier 
for "John dripks wine", the system enters the formula f6r 
"drinks", and draws inferences corresponding to each case 
sub-formula. In this example it will derive template^ like 
forms equivalent to, in ordinary English, "The wine is in 

"The wine entered John via an aperture'* and so on. 
The extracted templates express information already impli- 
citly present in the text, even though many of the^^"*are 

♦Wilks '73b, and in press 

4^ / 
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partial infejfences: ones that may not neceasarily be true. 

Com»o;<-sen*e inference rules are then brought down, 
which atttenptyby a sinple strategy, to construct the shor- 
test possible chain of rule-linked template forms from one 
containing an ambiguous pronoun, say, to one containing one 
of its possible referents. Such a chain then constitutes 
a solution to the ambiguity problem, and the preference 
approach assumes that the shortest chain is always the right 
one. So, in the,, case of "John dranK the wine on the table 
and it was good", the correct chain to "wine" uses the two 
rules: 

1 1. ((*ANI 1) ((SELF IN) (MOVE CAUSE)) (*RBAL 2) )"»• (1 (*JUDG) 2) 
or, in "semi-English", 

[animate 1 cause-to-move-in-self real-object-2]-^Cl *Judges 2} 
I 2. (1 BE (GOOD KIND))"^( (*ANI 2) WANT 1) 
or, again, 

Q is goodQ •^--*-C»"l-'"*^®~2 wants l] 

These rules are only partial, that is, they correspond 
only to what we may reasonably look out for in a given situation, 
not to what MUST happen. The hypothesis here is that under- 
standing can only take place on the basis of simple rules that 
are confirmed by the context of application. In this example 
the chain constructed may be expressed (using the above 
square bracket notation to contain not a representation, but 
simply an indication, in English, of the template contents) as: 
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[john dremk the wineJ 

forwards fjohn cau^es-to-move-in-self wineT 

inf. "'J T 
jjohn * judges wine] 

fjohn wants wine] 

good] 



backwards I r- 
inf. I ^ 



wine is 



[?it is 



good[ 



Teooplate 1 
» Tenplate 1 
by II. 

» line above 
by 12. 
Tenplate 3 



The assunption here is that no chain using > tother inference 

rules could have reached the "table" solution by using less ^ 

than two rules. 

The chief drawback of this system is that K:odings consis- 
ting entirely of primitives have a considerable amount of 
both vagueness and redundancy. For example, no reasonable 
codi.ng in terms of structured primitives could be expected 
to distinguish, say, "hammer" and "mallet".*" That may not 
matter provided th^ codings can distinguish importantly dif- 
ferent senses of words. Again, a template for the sentence 
"The shepherd tended his flock" would contain considerable 
repetition, eac^^. n6de of the template trying, as it were, to 
tell the whole story by itself. Whether or not such a system 
can remain stable with a considerable vocabulary, of say 
several thousand words, has yet to be tested. 

6, SOME COMPARISONS AND CONTRASTS 

In this section I shall compare and contrast, under some 
eight interconnected headings, the projects described in the 
body of thi notes. This is not easy to do, pajrticular ly when 
the present author is among the writers discussed, though 
that is easily remedied by the reader's making an appropriate 
discount. A more serious problem is that, at this stage of research 
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in artificial intelligenco and natural language, the jmost 
attractive distinctions dissolve on more detailed scrutiny, 
largely because of the lack of any precise? theoretical state- 
ment in most, if not all, the major projects. There are' those 
who think that it therefore follows that this is not the moment 
for any form of critical comparison in this field, and that 
no more is needed than a "positive attitude" towards all possible 
projects. Only those who feel that, on the contrary, any time 
is as good as any other for the discussion of intellectual 
differences in the hope of progress, should read on. 

It must be admitted, right away that the selection of 
projects discussed above, like Winograd's distinction between 
first and second generation systems, on which the selection 
^ was to some extent based, cannot be defended by any strict 
definitions: one that would, in this case, include all the 
projects described, and exclude all those of Winograd's 
"first generation". One might, for example, want to define 
second generation systems (in the study of natural alanguage 
within the AI paradigm) in some very general terms, such as 
thoare syistems which, (1) contain complex semantic ^tructurcs for thi 
representation of text that are signifj^antly different 
from the "surface structure" of the input, and (2) contain 
cognate structures representing conceptual and real world 
knowledge that is not explicitly present in the input text. 
Even so general a description of a "frames" type approach 
would not cover Charniak or Colby with the first point. 
Moreover, thu second point would certainly cover Winog^fad's 
work, as well as other first generation approaches, sd| it is 
clear, at the outsdt of any comparison, that there is not 
^oven a simple and unequivocal definition which covers all 
and only the projects to be compared. 



46* 



Level of Representation 

One Important line of current dispute among the second 
generation approaches concerns the appropriate level of re- 
presentation for natural language . On the one hand are 
those like Tolby, and apparently Charniak , ' Ikko hold that 
language can be in effect, s&lf repM^sC^nt^ijoug^ (or represented 
by what I called projections in Section 1) , while on the 
other hand# there are tho'se like Schank aifd^py«irf^]j*hcr hold' 



thaf'^e appropriate level of coni^utation jtor Tiif^rences 

about natulra language is in some r^q.^U^c<*d , or pxtmi t^tve , 

representation. I wrote "appears^in the caae^if of Cliarni^ik 

because he holds that his structuras are independent of. ariy 

particular level of representation > ^^ Tr^Tther , that^they 

could be realized at a number of levels of representjation , 

depending on the subject aroa, Howevet, there is no 

doubt that the representation in terms of predicates that 

he offers in his work appears to be in ono-to^one Correspondence 

with English words. 

The strongcjst low-level approach is undoubtedly that of 
Colby, who straightforwardly facus the enormous ifflll^>p 1 Tf^T** ' ^ 
problems involved if the structures are at the Engli^^ 
word level. It is important to realize that th^s dispute 
is ultimately one of degree, slnco no one would cXairo that 
every locution recognized by .^n int^^lligent ;\nalyser must 
be mapped into a "deep** reprosent-^tion. To take an eJctJC^m^ 
case , any snstem that mapped Good MOrning" into a deep 
semantic roprosentation before deciding that the correct 
response was alco "Good Morning* would be making a serious 
theoretical mistake. 

However, ^he most serious argument for a non-superficial 
representation is not in terms of the avoidance of mapping 
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difficulties, but in terms of theoretical perspicuity of the 
primitive structures, and this argument is closely tied to. the 
defence of semantic primitives in general, which is a large 
subject not to be undertaken here. One of the troubles about 
semantic primitives is that they are open to bhd defenses, 
which decrease rather than increase their plausibility. 
For example, some users of them for linguistic representation 
have declared them to have some sort of objective existence, 
and have implied that there is a "right set** of primitives 
open to empirical discovery. On that view the essentially 
linguistic charactGjf of structures of primitives is lost, 
for they then might as well be strings of binary numbers, or 
something equally opaque and non-linguistic. No groat deal 
of thought is required to set! that that simply could not be 
the case. What is the case is that there is a considerable 
amount of psychological evidence thr t people are able to 
recall either the actual words or the syntactic structure 
used. There is large literature on this subject, from 
which two sample references would be (Wcttler '73) and 
(Johnson-Laird *74) . 

These results are, of course, no proof of thu existence 
of semantic primitives, but they are undoubtedly supporting 
evidence of their plausibility, as is, on n different plane, 
the result from the encoding of the whole IJebster's Third 
International Dictionary at Systems Development Corporation, 
whor« It was found that a rank-orderud frequency count of 
the words used to define other words in that vast dictionary 

was a list (omitting "the** and "a*) which corresponded almost 

• ^ ✓ 

1 tem--f or-item to a plausjiblc list ojt semantic primitlvus, 
derived a priori, by thcfse^^^ftually concerned to code the 
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structure of word and sentence meanings. 

It is important to distinguish the dispute about level 
from the, closely connected, topic that I shall call the 
centrality of the knc^j^dge required by a language under- 
standing system • 

Centrality . 

What I am calling the centrality of certain kinds of 
Information concerns not its level of representation but 
its non-specificity: again a contrast can bo drawn between 
the sorts of information required by Charniak's system, on the 
one hand, and that required by Schank's and my own on the 
other. Charniyk's examples suggest that the fundamental 
form of information is highly specific* to particular situa- 
tions, like parties and the giving of presents, while the 
sorts of information central to Schank ' s and my own systems 
are partial assertions about human wants, expect?ition^ and 
so on, many of which ap^ so general as to be alDjpjHfcr'tracuous 
which, one might argue, is why their role in ttnder^J^nding 
has been ignored for so long. 

If I were .a reasonably fluent speaker of, say, German, 
I m ight well not understand a German conversation about 
birthday presents unlr^^^^^ had detailed factual information 
about how Germans organize the giving of presents , which 
might be considerably different from the way we do it. 
Conversely, of course, I might understand much of a techni- 
cal article about a subject in "VflTich I was an o^^fTort, even 
though I knew very little of the language in which it was 
writtoii. . * 

: — ' ■ 

♦In Charniak's most recent paper (1974), he gives much more 
general rules, such -:i3 his **rul^ of siani^ficant s\ib-action" • 

4.9 . 
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There are certainly considerAtions that tell for Charniak ' s 
approachi and it is perhaps a paradox that the sort of 
natural language under standor that would tend to confirm his 
assumptions would be ono concorned with discourse about, say# 
the details of repairing a motor car, where factual informa- 
tion is what is central, yet, ironically, Charniak has 

I " 

concentrated on something as general «ts. children's stories^ 
with their need of deep assumptions about hun^an desires and 
behaviour* 

In the end this difference may again turn out to be one 
of emphasis , and of what is most 'appropriate to di f f erent 
subject areas, though there may be a very genaral issue 
lurking somewhere here. It seems to me not a foolish question 
to ask whether much of what appears to be about natural 
language in AI r<$ooarch is in fsigt about language at all. 
Even if it is not thc^t may in no way detract from its 
valme* 

Newell (Moore, Newell '73) has argued that * 

* ^ 
work is in fact ^theoretical psychology", in which case 

it could hardly be r^ijsearch on natural language. when 
describing Winogrid's work earlior in the paper, I raised 
this question in a weak form by asking whcth*;ir his defini- 
tion of "pickup" had anything to do with the natural language 
use of the word , or whether it wao rather a description of 
how his system pick^id sctrethin^f up, a quite different matter. 

Suppose wo generalize this query somewhat , by asking 
the apparently absurd question of what would be wrong 
with calling, say , Charniak * s work an uscny on the 'Socio- 
Economic Behaviour of American Children Und^ir Stress?' ^ 
In the case of Charniak 's work thia is i f ace tious (Question, 



ask^jd only in order to make a point"? but with an increasing 
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number of systems in A.I. being designed not essentially to 
do research on natural language, but in order to have a natural 
language "front end" to a system th^t is essentially intended 
to predict chemical spectra, or play snakes and ladders or 
whatever, the question becomes a serious one. It seems to be 
a gpod time to ask whether wg should expect advance in under- 
standing natural lang,uage from those tackling the problems 
head on, olr*t?hose concerned to build a front end". It is 
clearly tjae case that any piece of knowledge whatever could 
be essential to the understanding of some story. The question 
is, does it'follow that the specification, organization ?nd 
f ormnlization of that knowledge is the study of language, 
becauae if ''it is tlj^n all human enquiry from physics and 
history to medicine is a linguistic enterprise. And, of 
course, that possibilUty has actually been entertained within 
c^rtain^^trains of ipodorn philosophy. 

Howcv^i I am not trying here to breathe fresh life into 
a philosophical distinction, botv/oon being about language 
and not being about language, but rather introducing a 
practical distinction, (which is also a consideration in 
favour of opting, as I hav^^, to work on very general and 
central areas of knowledge) between specific knowledge, and 
central knowledge without which a syst j ^m could not bo 
said to understand the language at ^all . For example, I 
might know nothing of tht ^rrangumk^nt of American birthday 
parties, but could not be accused of not understanding % 
English even though I failed to unduX-a4;;.and some particular 
children's story. , Yet, if I did not havu availnble some 
very general partial inference such as the onu people being 
hurt and falling, or on»j about peoplw endeavouring to^ 
possess things that they want, then it is quite possible 
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that my lack of understanding of quite simple sentences would 
cause Observers to think that I did not understand English. 
An interesting and difficult question that then arises is 
whether those who concentrate on central and less central 
areas of discourse could^ in principle, wold their bodies 
of inferences together in such a way as to create a wider 
•ystom; whether, to put the matter another way , natural 
language is a whole that can be built up from parts? 

Phonomeno logical level 

Another distinction that can bo confused with the central^ 
specific one is that of the •'phenomenological levels" of 
inferences in an understanding system. I mean nothing 
daunting by the phrase: consider the action mating which 
is, as a matter of anatomical fact, quite often an act of 
bringing the bones of my ulna and radius (in my arm) close to 
that of my lower mandible (my jaw). Yut clearly, any system 
of common sense inferences that considered such a truth 
when reasoning about eating would bu making a mistake. One 
might say that the phenomenological level of the analysis 
was wrong even though all the infarenctis it made were true 
ones. The same would bo truo of any A.I. systt^m that made 
everyday inferences about physical objects by confifidering 
their quantum structure. 

Schank's analysis cf oating contains the information that 
it is done by moving the hands to the mouth, and it might be 
argued that c;von this is going too far from the **moaning** 
of eating, whatever that may be,>^towards genc^rally true 
information about the act which, if always inferred about ^ 
all acts of eating, will carry the system unmanageably far. 
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There is no denying that this sort of information migh^- 
be useful to have around somowherw»> that, in Minsky's terms', 
the "default" value of the instrument for eating is the 
hand brought to the mouth, so that, if we have no contrary 
information, then that is the way to assume that any given 
act of eating was performed. Nonetheless, there clearly is 
a danger, and that is all I am drawing attention to here, 
of taking inferences to . a phenomenological l&vel beyond that of 
common sense. A clearer case^_J.n ihy view, would bo Schank's 
analysis (1974a) of ment/il activity in which all actions, 
such as kicking a ball, say, arc preceded by a mental action 
of Conceiving or deciding to kick a ball. This is clearly 
ft level of analysis untrue to cojpmon sense, and which can' 
have onTy harmful effects in a system intended to mimic 
common sense reasoning and understanding. 

Decoupling 

Another ceneral issue in dispute concerns wJ^at I shall 
call decoupling , which is whether or not the actual j parsing 
of text or dialogue into an "understanding system" is essen- 
tial. Charniak and Minsky believe that this initial "parsing" 
can be' ef f feiCtive'ly decoupled from the interesting in-fcrential 
work and simply assumed. But, in my view, that is not so, 
because many of the later inferences would actually hrave to be 
done already, in order to h^vc achieved the initial parsing. 
For example, in analysing "ho shot hur with a colt", we 
cannot ascribe any structure at all tintil we can make the 
inference that guns rnther than horses ar»j instruments for 
shooting, and so B^jfh a sertence cannot* be insextod into an 
inference-but-no-pnrsing struc^ur: (as in (AT JOHN STATION)) 
without asi»uming that language docs not h^ve one ot its 

J* • 
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essential characteristics, namely systcrfiatic ambiguity . 
As I argued at tho beginning of the notcsi if one allows 
represont^itional structure's to h^.ve significance cjuitc indu- 
pondent of their application/ then one may not be in a situa- 
tion essentially different from that of the logicim who 
simply asserts that such-and-such is the 'right structure" 
of some scntoncQ* 

Also, tho infcroi)co3 required to resolve word sense ambi- 
guities, and those required resolve pronoun referonco * ^ 
problems, are not of different typesj often the two problvams 
occur in a simple sentence and must be resolved together. 
But Charniak's decoupling his the effect, of completely 
separating these two closely related linguistic phonomona 
in what seems to me nn unrealistic manner. His system does 
infarencing to resolve promoun ambigui tilths , while sense 
ambiguity is presumably to bo done in the future by some 
other, untimately recouplcd, system.* 

Another way of pointing up the differjtice between the > 
attitudes of second genorition systems to decoupling, in 
relation, to the first generrition, is by describing the role 
of syntax analysis in them*. As w: saw, syntax was tho^hjoart 
of Winograd's syytcm, but both levjls of frame approach dis- 
count syntax analysis, though for v ry different reasons: 
Charniak does ©o because it Is p rt of the initial parsing 
from which his inferenti-^l work has be^n decoupled . Schank 
and I do so because we beli^jive s mantic analysis tc be funda- 
mental, and that in an c*ctual impluraent ation the results of 
syntactic analysis c^n all achi^jved by a sufficiently 
powerful Semantic anolyser. And this last asiiumrtion ih 

— — ■ ■ — '■ — ■ . \ 

♦Although Charniak wiuld argue that sense ambiguity could be 
introduced into" hlff systeiB in Its pr' Sent form.' 
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confirmed by the limited degree of success that the two seman- 
tic analysers have actually achieved in operation. - 

Availability of surface structure ^ ^ 

An issue close to that of the" appropriate level oi^ repre- 
sentation in a system is that of the availability of the sur- 
face structure of the language analysed? or, to put it more 
crudely, the availability . during subsequent analysis of the 
actual words b^ing analysed. These ard clearly available in 
Colby, andiare indirectly available in. Wlnograd * s ^nnd my own 
system, but Shcank makes a point of the importance of their 
non-availabili ty > on the grounds that an ideal representation 
shoulld be totally indepc^ndent of the input surface structure 
and words. There are both theoretical and prjCctical aspects 
to this claim of Schank's: in th imit, the order of th^ 
sentences a text is part of its surface structure, and 
presumably it is not intended to abandon this "superficial 
inf ormatiotv" . ^ In omv of his recent papers (1974b) Schank 
seems to have accepted some limitation on the abandonment 
of surface striictiire. 

The other, prcctical point concerns tht, form of represents- 
tion employed: in the (1973) implementation of Schank 's sys- 
tem using an analys.ir of input toxt, mec^ory and ^ generator 
of rf^«ponses, it was intended that nothing should be trans- 
jferred from the input pro-am* to the output program except 
a represc;ntation coded in the structures of primitives dis- 
cus sed^ ear lieT* . The question that arises is, can that 

4 

Structure specify and distinguish word-senses adequately 

without transferring in-formation specifically associated 

with the input word? Schank clearly believes the answor^to 

* This point is to some extent hypothetical since, as we saw, 
Schank' s conceptualizations still do contain, or appear to con- 
tain, many surface items; in particular nouns, |djectives an< 
adverbs. However, this is a transitional matter and thev. 
in the course of replacement , as noted, by non-superf ici/al items. 
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this question is yes, but that cannot be considered established 
by the scale of^ computations yet described in print. 
^ Ai^suitable environinent in which to consider the question 
is that of translatidti^ from one language to another: suppose 

are an^ilysing a sentence containing the word "nail", 
meaning a physical object.* It is clear that the translation 
of that 'Vord into French should not be the same as the trans- 
lation for "scfewy^r "peg". Yet is it plausible th^t any 
discriprtion of the function of these three entities entirely 
in terms of semantic primitives , and without any explicit 
mention of the word name and its connection to its French 
equivalent, will be sufficient to ensure that only th'e.^ 
right match is made? 

replication ^ - 

Thls^oint is a generalisation of the last two,^nd 
concerns the 9lay in which different systems display, in the 
structures they manipulate, the actual ^procedures of appli^ 
cation of those structures to iA^ut teAt or dialogue. This 
is a matter d^^^^r^nt from both that of the availability 
of the surf ace \tructure , and of a com^ter ^ J^plomentation 
of the system. In the case of» Colby's patterns for example 
the form of tTheir application to the input E^nglish isVclear, 
even though the mlitchin^ invalvisd ^could be achieved by many 
different implementation algoriynms. In the case of my own 
system, I h/^^d the same to be true the tcmpilate structures, 
even thotfgh by the time th^^^^jjiput has reached tne canonical 
templatye form it is considerably di^fferent from. J;he input^ 
surfacA stptf^ture. The system ^t thevoxtreme end of any 




scale of perspicuity of application isLWijiograd * s , where 
the procedural notation, by its nature, tends to make clear 
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the way in which the structures are applied. AT the other 
end are the systems of Schank and Charniak, where no appli- 
cation is specified," which means that the representations are 
not only compatible with many implementation algorithms, which 
does/^io^ master , but arc also compatible with many systems 
ofr lirv^uistic rules, whose specification is an essential piece 
owjfliquiry, and whose subsequent production may cause the 
balft: system to be fundamentally different. 

English prepositions will serve as an example: in Schank' s 
case notation there is no indication of how the cas^ discrinina- 
s are actually to bo applied to English prepositions in 
•\ So, for example, the preposition "in" can correspond to 
containment case, time location, and spatial location, 
among others. As wj/ saw earlier, the discrimination, involved 
in actual analysis is a matter of specifying very delicate 
semantic xules ranging over the basic semantic structures 
emplo\yed. Ifrd^e^, the structures and case system themselves 
seem to me to b^ essentially dependent on the nature and 
applicability of gtfcii^rulj/s* , and so this application of the 
system should have an obvious place in the overall structures. 
It is not vseomthing to be delegated to a mere "implementation". 
If enough of the linguistic intractables ** of English analysis 



* This is not meant to be just bland assertion. I have written 
at some length on ^he relations between application and the 
theoretical status of linguistic theories in (Wilks '74) 

** The differences between Minsky's (1974) notion of "default 
value" and what I have called ''pr^jf erence" Cc*.n be pointed up An 
te^ms of application. Minsky suggest '"gun" as the default value 
ofpthe instrument of the action of shooting, but I would claim 
thart, in an example like the earlier "He shot her wi^h a colt", 
we 4aeed to be able to see in the structure assigned wiether or 
JTOt what is offered as the apparent instrument is in fact an 
instrument and whethe^ it is the default or not. In other words, 
we need sufficient structure of application to see not only that 
"shooting" prefers an insttrument that is /gun, but also why it 
willj,choose the sense of "colt" that is a gun rather than the 
one which is a horse. 
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were to be delegated out of the representation, A.I, would 
be offering no more to the analysis of natural language than 
the logicians who proffer the predicate calculus as a 
plausible structure for English. 



develop a view that is considerably stronger than this 'appli 
cation'one: in his view the control structure of an under- 
standing program is itself of theoretical significance, for 
only in that way, he believes, can natural language programs 



Forwar inf erenpe 

Another outstanding dispute concerns whether one should 
make massive forward inferences as one goes through a text, 
keeping all one*s expectations intact, as Charriiak and Schank 
hold, or whether, as I hold, one should adopt some 'laziness 
hypothesis' about understanding, and geriorate deeper inferences 
only when the system is unable to solve, say, a referential 
problem by more superficial methods. Or, in computer terms, 
should a system be problem or date-driven? 

Although Shank sometimes writes of a system making "all 
possible" inferences as it proceeds through a toxt, this is not 
in fact the heart of the dispute , since no one would want 
to defend any strong definition of the term '*all possible 
inferences" . Charniak's argument is that, unless certain 
forward inferences are made during an analysis of, say, a 
story forward inferences, that is, that are not problem-driven j 
not made in response to any particular problem of analysis 
then known to the system then, as ^. matter of empirical fact, 
the system will not in general be able to solve ambiguity 
or reference problems that arise later, because it will 
never in fact be possible to locate (while looking backwards 



In some of his more recent writings Winograd has begun to 



of great size and complexity remain perspicuous. 
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at the text, as it were) the points where those forward 
inferences ought to have been made. This is, in very crude sum-- 
oary, Charniak's case against a purely probleo-drivcn infer- 
encer in a natural .language understander . 

A difficulty with this argument is the location of an example 
of text that confirms the point in a non-contentious manner. 
Charniak has found an excorpt from a book describing the 
life of apes in which it is indeed hard to locate the reference 
of a particular pronoun in a given passage. Charniak 's case 



is that it is only possible to do so if one has made ^erttin ^ 
Xnon-problem occasioned) inferences earlier in the story. 
But a nunxber of readers find it- quite hard to refer that 
particular promoun anyway, which might siggost that the 
text was simply badly written. 

Wiis is a difficult matter about which to be precise; It 
woiild ht^y^ossible , for example, to agree with Charniak 's argu- 
ment HndVstill construct a purely problem-driven inferencer 
on tMe giip^nd that, at the moment, this is the only way one 
can (topeXritJi the (yast majority of inferences for under- 
standing, since any system of inferonces made in response to 
no particular p roblem iri the toxt is too hard to control In 
practice. Indeed, it is noticeable thnt the most recent papers^ ^ 
of Schank (1974a and 1974b) ^nd Charniak (1974) have been 
considerably loss forward-inf cr^nco oriented than oarlier 
ones* . 

, This dispute is pr.rhaps only one of degree, and about the 

possibility of defining a degro* of forward inference that aids 

the solution of later semantic problums without going into 

unnecessary depth**. This might be an aro^ whcrr, psychological 

investigations would be of enormous help^ to workers in AvIt 

♦ A particularly intc^resting withdrawl of a strong forward in- 
ference thesis is hidden away on p. 283 of (Rit^ger '74) but has 
been located by the k^jen eye of Charniak 

** which may be no mor^ than a fl^chological rGstat..mcnt of 
what used to be callud (Hayes, "71) (Sandewall '12) the frame 
problem, (no relation, P.E.) \ 



59. 



The justification of systems 

Finallyr one might usefully, though briefly, contrast the^ 
different modes of justification implicitly appealed to by the 
systems described earlier in this paper. These seem to me to 
reduce to four: 

(i) In terms of the power of the inferential system employed 
This form of justification has underlain the early predicate 
calculus-based language programs , and is behind Hayes* (1974) 
recent demand that any formalism for natural language analysis 
should admit of a set theoretic semantics, in the Tarskian 
sense, so as to gain "intellectual respectability "7\^as he puts 
it. The same general typo of justification is appealed to 

in some degree by systems with PLANNER-type formalism*. 

(ii) In terms of the provision and f ormalisation , in any 
terms including English, of the sorts of knowledge required 
to underi^^and araas of discourse. 

(iii) In terms of the actual performance of a system, 
implemented on a computer, at a task agreed to demonstrate 
understanding . 

(iv) In terms of the linguistic and'-or psychological 
plausibility of the proffered system of representation. 
Oversimplifying considerably, one might say that Charniak's 
system appeals mostly to (ii) and somewhat to (i) and (iv) j 
Winograd's to (iii) and somewhat to the other three categories^ 
Colby's (as regards its natural language, rather than 
psychiatric, aspects) appeals almost entirely to (iii), 

and Schank's and my own to differing mixtures of (ii) , 
(iii) and (iv) . 

In the end, of course, only (iii) counts for empiricists, 
but there is considerable difficulty in getting all parties to 
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agroo to the tdrms of a test.-^ A cynic Bight say that, in the 
end, all these systems analyse the »entencep that thoy analyse 
or, to put the same point a little more theoretically, there 
is a sense in which systems, those described here and those 
elsewhere, each defino a natural language , namely the one 
to which it applies. Thp^ difficult question is the extent to 
which those many and, Small natural languages resemble English. 



7. COHCLUSION 

The last section stressed areas of current disagreement, but 

there would, if votes were taken, be considerable agreement 

among A.I. workers on natural language^bout whore^'^^ >arge 

problems of the immediate future are: the need for a yood memory 

I 

model has been stressed by Schank (1974a) , and many would 
add the need for an extended procedural theory of texts, rathier 
than of individual example sentences, and for a more sophisti- 
cated theory of reasons, causes, and motives for use in a 
theory of understanding.' *Many might also be persuaded to 
agree on the need to steer between the Scylla of ^ trivial 
first generation implementations and the Charybdis^ of 
utterly fantastic ones. By the latter, I mean projects that 
have been sei^iously discussed, but never implemented ^o*^ 
obvious reasons, that would, say, onablu a diJ^j^logue progiaca 
to discuss whether or not a participant in ^ given story 
"felt guilty", and if so why. ; ^ 

The last disease has sometimes had ns a major symptom an 
extensive use of the word "pragmatics" (though this can also 
indicate quite benign conditions in other Cy4ses) , along with 
the implicit claim that "semantics has been solved, so we 
should get on with the pragmatics". It still needs repeating 
that there is no sense whatever in which the semantics pf 
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natural language hpLB pnon solved. it is still the enormous barrier 
it has always been, even if a few dents in its surface are 
beginning to appe<^r here and there. Even if w.e istick to the 
simplest examples i that present no difficulty to the 
human reader — "anjb it must bo admitted th?«.t it has been one of 
the persistent fa^ults of thC: A. I. paradigm of language that 

it has spent too ;much time on puzzles examples there are 

still great difficulties both systematic and linguistic. 

An example of the former would be the development of a 
synamic system of undersTtanding texts or stories that had 
any capacity to recover aftpr having its expectations satisfied 
and then, subsequently, frustrated.. At present no system of 
the sort described, whether of demons, preference or whettever, 
has any such capacity to recover. The situation is qui/e 
different from that in a dialogue, as in Winog^rad's sy/tera, 
where, on being given each now piece of information, l[he 
system checks it against what it knows, to see if it/is being 
contradicted, and then behaves in an appropriately/puzzl-ed 
way if it is. In frame or "expectation" system^X^t is all 
too easy to construct apparently trick, buj^asically 
plausible, examples that satisfy what wjv^boing i«okea for 
and then overturn it. That possibility is alre^tt^-bu-ili^ int^_ 
the notion of frame or expectation. An example of Phil Hayes 
against my own system will set-vc; consider "The hunter licked 
his gun all over, and the stock tasted especially good". 
What is meant by "sfaock" is clearly the stock pieee of the 
gun, but any preference system liko mine that considers the 
two senses of "stock", and sees that an edible, soup, 
sense of "stock" is the preferred object of the action ' 
"taste", will infallibly opt for the wrong sense. Any frame 
or expectation system is prone to the same general kind of 
counter-example. * 
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In particular cases like this it is easy to suggost what 
might bo done: hero wo might suggest jo. proforonco attached 
to the formula for anything that was essentially part of 
another thing (stock « 'part of gun" in this case), so that 
a local search was made whenever the "part-of" entity was 
mentioned, and the satisfaction of that search would always 
be the overriding preference. But that is not the same as 
a general solution to the problem-, which uSed to bo called 
that of "topic" in the computational semantics of the 
Fifties. There are no solutions tp this problojp available 
here and now, though some suggestions have been mlde by 
Abelson (1974) and McDermott (1974). 

A closeiy rol?.ted, but equally intractable, problem is 
that of how to combine highly ' specif ic factual information 
within a general semantic structuring. Systems like Charniak 
arc, as we saw, concerned with specific rather than concep- 
tual information, but there are quite simple "semantic speci- 
ficity" problems and one could not reasonably expect to 
bo tackled oven in a system devoted to the handling of 
facts, as can be seen by contrasting the sentences: 

The deer came out of thi wood. 

The grub cimt/ out of the wood, 
whore we might safely assumu that readers would assign quite 
d-.fferent senses to "wood" In th..- two cases simply on the 
basis of the two different agents. No-one, to my knowledge, 
has suggested any general method foV tnckling such elementary 
examples. 

Cut, to finish on thu bright side, it is important to 
stress that ^ere ia indeed an A. I. paradigm of language 
understanding in existence*, one that embraces first and 

* One of the very few acknowledgements of this fpct, of the 
possibility of an A.I. paradigm of language, from a linguist 
is (Fillmore '74) . 
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second generation eipjproochos and which goes«bac^i 1 suggested, 
to a conil3eriWe aiouht oB earlier work in com^utatlQnal 
linguistics. It can bo distinguished by a catalogue of 
neglect >y conventional linguistics that can bo suaoarised 
under throe heads: 

(i) Theories of language must have procedural application 
to the subject matter that could in principle result, in com- 
puter application and subsequent empirical test. 

(ii) r theories of language must deal with it in a commu- 
nicative context, one amenable to ompiri*cal assessment . Merely 
sorting, as genorativet theories wore designed* to* do, is not 
enough. 

(Hi) theories of language must nlso bo , in clear sense , 
theories of the f ormalisation and organization of knowlodge. 
If they «tro not then we can know in advance that they can 
never tackle the problem of language understanding. 

/ 

A NOTE ABOUT THE REFERENCES: 

The references are given only for completeness. It is 
not of course expected that those attending the course should 
be familiar with them all. However, it would be good to 
have glanced at the details 6t an article by each 6f the 
authors surveyed. In the abbreviated form of reforonces below. 
MOD and AIJC3 each contain articles by 3 of the 4 authors 
(not the same 4) . 

In order to compress the reference list the following 
abbreviations for colloctions of articles will used* 

AIJC3 AdvancCid papers of the Third Internationril Joint 
Conference on artificial Intelligence, Stanford, 
CaLif. 197 3 
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ASSB Proceedings of the Sumroor Conference of the Society 

for Artificial Intolligence and Simulation of Behaviour, 

University of Sussex* 1974 
TEDD Proceedings of the First International Conference on 

^4achine Translation, National Physical Laboratory , 

Teddington, 1961 ThMSO, Lotrdon, 1961) . 
ACL Proceedings of the Conference of the Association for 

Computational Linguistics, Amherst, Mass. 1974. 
CAST Memoranda from the Istituto per gli studi Semantici e * 

Cognitivi , Castagnola, Switrtrland 
MITAI Memor^uvda from the Artificial Intelligence Laboratory, 

Massachusetts Institute of Technology* 
SUAIM Memoranda from the Artificial Intelligence Laboratory, 

Stanford University, Stanford, Calif. 

SRITN Technical notes from tho Stanf ord^Resoarch Institute , •^^ 

* 

Menlo Park, Calif. 
MOD Papers in C omputer MQdels of Thought^ and Language , 

ed. by Schank and Colby, (Freeman, San Francisco, 1973) . 
(Abclson '73^ 

Abelson, R. P., Tho Structure of Btili€;f Syfttums, in MOD. 
(Abelson '74) 

Abelson, R. P., Discourse Headers, to appear in Papers in 
Memory of J . Carbonell . 

(Colby, Parkison *74) 

Colby, *K. M., Pnrkison, R. C, Pattern-^Matching Rules for the 

Recognition of Natural Language Dialogue Express ions , « 

in American Journal of Computational Linguistics , Vol. 1, No. 

(Colby, Hilf '73) 

Colby, K. M. , Hillh"^» E>., Multidimensional Analysis in 
Evaluating a Simulation of Paranoid Thought , in 3*0 AIM- 194 . 

(Colby et al * 71) 

Colby, K. M., et al, Artificial Paranoia, Artificial Intolli- 
qence 2 . 
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(Enea, Colby "^73) 

Enea, H. , ColbVr K. M., Idioloctic Language-Analysis for 
Understanding Doctor-Patient Dialogues, AiaC3. 

{Fillmore '74)' 

Fillmore, C, Suamer School lectures. University of Pisa, Italy. 
(Goldman '74) 

Goldman, N. M., Cumputer Generation of Natural Language from 
a Deep Conceptual Base, CAST No. 2. 

(Halliday '70) 

Hjalliday, M* A. K.^ in Foundatior^^ of Language . 

/ 

(Hayes '71) 

Hayes, P* J., The Frame Problem and Related Problems in 
Artificial* Intelligence, in SUAIM^53 

(Hayes •74) ( 

Hayes, P. J., Somo Problems and Non^rbblems in Representation 
Theory, in AISEv \ 

(Hays ^64) 

Hays, D, , Dependency Theory: A Formalism and Somo Observations, 
Language , Vol* 40* 

(Herskovits '73) 

Hcrskovits, A., The generation of French from a semantic 
represent at! on , SU AIM- 212 • 

(Johnson-Laird ^74) ! 
Johnson-Laird, P., Memory for words, Notur^, Vol. 251. 

(McDerPiott ^74) 

McDermott, D. V., Assimilation of New Information by n 
Natural Language Und^rs tandin.g System, MITAI, Tr-291. 

(Minsky • 74) 

Minsky, fl. , Frame systems » unpublishqid MSS^U-^from Massachusetts 
Institute of Technology. 

(Rieger '74) 

Riogor, C. J.^ ConceptUcil Memory, Stanford University Ph.D. 
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